Method and apparatus for performing virtual pullback of an intravascular imaging device

ABSTRACT

The present disclosure provides a method of simulating an intravascular procedure in a virtual environment. The method includes displaying information from a first view and a second view simultaneously. The first view contains virtual representations of an anatomical region of a human body and an intravascular imaging device disposed in the anatomical region. The second view contains a cross-sectional image of a segment of the anatomical region corresponding to a location of the intravascular imaging device. The method includes moving, in response to a user input, the virtual representation of the intravascular imaging device with respect to the virtual representation of the anatomical region. The method includes updating the cross-sectional image as the virtual representation of the intravascular imaging device is being moved. The updated cross-sectional image corresponds to a new location of the intravascular imaging device.

TECHNICAL FIELD

The present disclosure relates generally to intravascular imaging, andin particular, to a method and apparatus for performing a virtualpullback of an intravascular imaging device.

BACKGROUND

Intravascular imaging is widely used in interventional cardiology as adiagnostic tool for assessing a vessel, such as an artery, within thehuman body to determine the need for treatment, to guide intervention,and/or to assess its effectiveness. An imaging system such as anintravascular ultrasound (IVUS) system uses ultrasound echoes to form across-sectional image of the vessel of interest. Typically, IVUS imaginguses a transducer on an IVUS catheter that both emits ultrasound signals(waves) and receives the reflected ultrasound signals. The emittedultrasound signals (often referred to as ultrasound pulses) pass easilythrough most tissues and blood, but they are partially reflected bydiscontinuities arising from tissue structures (such as the variouslayers of the vessel wall), red blood cells, and other features ofinterest. The IVUS imaging system, which is connected to the IVUScatheter by way of a patient interface module, processes the receivedultrasound signals (often referred to as ultrasound echoes) to produce across-sectional image of the vessel where the IVUS catheter is located.

Traditionally, an IVUS run is performed when an IVUS catheter is pulledback from a blood vessel. While the IVUS catheter is being pulled back,the transducer on the IVUS catheter captures cross-sectional ultrasoundimages of the blood vessel at various locations of the blood vessel.These images are saved to an IVUS system and may be reviewed andanalyzed by a physician later. However, the physician may not know wherein the blood vessel a particular ultrasound image is taken. As such,even if the physician spots a problem on the ultrasound image, it may bedifficult for him/her to perform an accurate diagnosis because the bloodvessel location corresponding to the particular ultrasound image may beunavailable.

Therefore, while conventional methods and apparatuses for performingintravascular imaging are generally adequate for their intendedpurposes, they have not been entirely satisfactory in every aspect.

SUMMARY

One aspect of the present disclosure involves a method of simulating anintravascular procedure in a virtual environment. The method includes:displaying information from a first view and a second viewsimultaneously, wherein the first view contains virtual representationsof an anatomical region of a human body and an intravascular imagingdevice disposed in the anatomical region, and wherein the second viewcontains a cross-sectional image of a segment of the anatomical regioncorresponding to a location of the intravascular imaging device; moving,in response to a user input, the virtual representation of theintravascular imaging device with respect to the virtual representationof the anatomical region; and updating the cross-sectional image as thevirtual representation of the intravascular imaging device is beingmoved, wherein the updated cross-sectional image corresponds to a newlocation of the intravascular imaging device.

Another aspect of the present disclosure involves an electronicapparatus configured to perform a virtual pullback of an intravascularimaging device. The electronic apparatus includes: a screen configuredto display an output to the user; a memory storage component configuredto store programming code; and a computer processor configured toexecute the programming code to perform the following tasks: displaying,on the screen, information from a first view and a second viewsimultaneously, wherein the first view contains virtual representationsof an anatomical region of a human body and an intravascular imagingdevice disposed in the anatomical region, and wherein the second viewcontains a cross-sectional image of a segment of the anatomical regioncorresponding to a location of the intravascular imaging device; moving,in response to a user input, the virtual representation of theintravascular imaging device with respect to the virtual representationof the anatomical region; and updating the cross-sectional image as thevirtual representation of the intravascular imaging device is beingmoved, wherein the updated cross-sectional image corresponds to a newlocation of the intravascular imaging device.

Yet another aspect of the present disclosure involves an apparatus thatincludes a non-transitory, tangible machine-readable storage mediumstoring a computer program. The computer program containsmachine-readable instructions that when executed electronically bycomputer processors, perform: displaying, on a touch-sensitive screen,information from a first view and a second view simultaneously, whereinthe first view contains virtual representations of an anatomical regionof a human body and an intravascular imaging device disposed in theanatomical region, and wherein the second view contains across-sectional image of a segment of the anatomical regioncorresponding to a location of the intravascular imaging device; moving,in response to a user input, the virtual representation of theintravascular imaging device with respect to the virtual representationof the anatomical region; and updating the cross-sectional image as thevirtual representation of the intravascular imaging device is beingmoved, wherein the updated cross-sectional image corresponds to a newlocation of the intravascular imaging device.

Both the foregoing general description and the following detaileddescription are exemplary and explanatory in nature and are intended toprovide an understanding of the present disclosure without limiting thescope of the present disclosure. In that regard, additional aspects,features, and advantages of the present disclosure will become apparentto one skilled in the art from the following detailed description.

BRIEF DESCRIPTIONS OF THE DRAWINGS

Aspects of the present disclosure are best understood from the followingdetailed description when read with the accompanying figures. It isemphasized that, in accordance with the standard practice in theindustry, various features are not drawn to scale. In fact, thedimensions of the various features may be arbitrarily increased orreduced for clarity of discussion. In addition, the present disclosuremay repeat reference numerals and/or letters in the various examples.This repetition is for the purpose of simplicity and clarity and doesnot in itself dictate a relationship between the various embodimentsand/or configurations discussed.

FIG. 1 is a schematic illustration of an intravascular ultrasound (IVUS)imaging system according to various aspects of the present disclosure.

FIG. 2 is a schematic drawing depicting a medical sensing systemincluding a multi-modality processing system according to one embodimentof the present disclosure.

FIG. 3 is a schematic drawing depicting a medical sensing systemincluding a multi-modality processing system according to anotherembodiment of the present disclosure.

FIG. 4 is a diagrammatic perspective view of an aspect of the medicalsensing system of FIG. 3, namely, the multi-modality processing system.

FIGS. 5-8 are user interfaces for performing a virtual pullback of anintravascular imaging device according to various aspects of the presentdisclosure.

FIG. 9 is a flowchart illustrating a method for performing a virtualpullback of an intravascular imaging device according to various aspectsof the present disclosure.

DETAILED DESCRIPTION

For the purposes of promoting an understanding of the principles of thepresent disclosure, reference will now be made to the embodimentsillustrated in the drawings, and specific language will be used todescribe the same. It is nevertheless understood that no limitation tothe scope of the disclosure is intended. Any alterations and furthermodifications to the described devices, systems, and methods, and anyfurther application of the principles of the present disclosure arefully contemplated and included within the present disclosure as wouldnormally occur to one skilled in the art to which the disclosurerelates. For example, the present disclosure provides an ultrasoundimaging system described in terms of cardiovascular imaging, however, itis understood that such description is not intended to be limited tothis application, and that such imaging system can be utilized forimaging throughout the body. In some embodiments, the illustratedultrasound imaging system is a side looking intravascular imagingsystem, although transducers formed according to the present disclosurecan be mounted in other orientations including forward looking. Theimaging system is equally well suited to any application requiringimaging within a small cavity. In particular, it is fully contemplatedthat the features, components, and/or steps described with respect toone embodiment may be combined with the features, components, and/orsteps described with respect to other embodiments of the presentdisclosure. For the sake of brevity, however, the numerous iterations ofthese combinations will not be described separately.

There are primarily two types of catheters in common use today:solid-state and rotational. An exemplary solid-state catheter uses anarray of transducers (typically 64) distributed around a circumferenceof the catheter and connected to an electronic multiplexer circuit. Themultiplexer circuit selects transducers from the array for transmittingultrasound signals and receiving reflected ultrasound signals. Bystepping through a sequence of transmit-receive transducer pairs, thesolid-state catheter can synthesize the effect of a mechanically scannedtransducer element, but without moving parts. Since there is no rotatingmechanical element, the transducer array can be placed in direct contactwith blood and vessel tissue with minimal risk of vessel trauma, and thesolid-state scanner can be wired directly to the imaging system with asimple electrical cable and a standard detachable electrical connector.

An exemplary rotational catheter includes a single transducer located ata tip of a flexible driveshaft that spins inside a sheath inserted intothe vessel of interest. The transducer is typically oriented such thatthe ultrasound signals propagate generally perpendicular to an axis ofthe catheter. In the typical rotational catheter, a fluid-filled (e.g.,saline-filled) sheath protects the vessel tissue from the spinningtransducer and driveshaft while permitting ultrasound signals to freelypropagate from the transducer into the tissue and back. As thedriveshaft rotates (for example, at 30 revolutions per second), thetransducer is periodically excited with a high voltage pulse to emit ashort burst of ultrasound. The ultrasound signals are emitted from thetransducer, through the fluid-filled sheath and sheath wall, in adirection generally perpendicular to an axis of rotation of thedriveshaft. The same transducer then listens for returning ultrasoundsignals reflected from various tissue structures, and the imaging systemassembles a two dimensional image of the vessel cross-section from asequence of several hundred of these ultrasound pulse/echo acquisitionsequences occurring during a single revolution of the transducer.

FIG. 1 is a schematic illustration of an ultrasound imaging system 50according to various aspects of the present disclosure. In someembodiments, the ultrasound imaging system 50 includes an intravascularultrasound imaging system (IVUS). The IVUS imaging system 50 includes anIVUS catheter 52 coupled by a patient interface module (PIM) 54 to anIVUS control system 56. The control system 56 is coupled to a monitor 58that displays an IVUS image (such as an image generated by the IVUSsystem 50).

In some embodiments, the IVUS catheter 52 is a rotational IVUS catheter,which may be similar to a Revolution® Rotational IVUS Imaging Catheteravailable from Volcano Corporation and/or rotational IVUS cathetersdisclosed in U.S. Pat. Nos. 5,243,988 and 5,546,948, both of which areincorporated herein by reference in their entirety. The catheter 52includes an elongated, flexible catheter sheath 60 (having a proximalend portion 64 and a distal end portion 66) shaped and configured forinsertion into a lumen of a blood vessel (not shown). A longitudinalaxis LA of the catheter 52 extends between the proximal end portion 64and the distal end portion 66. The catheter 52 is flexible such that itcan adapt to the curvature of the blood vessel during use. In thatregard, the curved configuration illustrated in FIG. 1 is for exemplarypurposes and in no way limits the manner in which the catheter 52 maycurve in other embodiments. Generally, the catheter 52 may be configuredto take on any desired straight or arcuate profile when in use.

A rotating imaging core 62 extends within the sheath 60. The imagingcore 62 has a proximal end portion 68 disposed within the proximal endportion 64 of the sheath 60 and a distal end portion 70 disposed withinthe distal end portion 66 of the sheath 60. The distal end portion 66 ofthe sheath 60 and the distal end portion 70 of the imaging core 62 areinserted into the vessel of interest during operation of the IVUSimaging system 50. The usable length of the catheter 52 (for example,the portion that can be inserted into a patient, specifically the vesselof interest) can be any suitable length and can be varied depending uponthe application. The proximal end portion 64 of the sheath 60 and theproximal end portion 68 of the imaging core 62 are connected to theinterface module 54. The proximal end portions 64, 68 are fitted with acatheter hub 74 that is removably connected to the interface module 54.The catheter hub 74 facilitates and supports a rotational interface thatprovides electrical and mechanical coupling between the catheter 52 andthe interface module 54.

The distal end portion 70 of the imaging core 62 includes a transducerassembly 72. The transducer assembly 72 is configured to be rotated(either by use of a motor or other rotary device or manually by hand) toobtain images of the vessel. The transducer assembly 72 can be of anysuitable type for visualizing a vessel and, in particular, a stenosis ina vessel. In the depicted embodiment, the transducer assembly 72includes a piezoelectric micromachined ultrasonic transducer (“PMUT”)transducer and associated circuitry, such as an application-specificintegrated circuit (ASIC). An exemplary PMUT used in IVUS catheters mayinclude a polymer piezoelectric membrane, such as that disclosed in U.S.Pat. No. 6,641,540, hereby incorporated by reference in its entirety.The PMUT transducer can provide greater than 50% bandwidth for optimumresolution in a radial direction, and a spherically-focused aperture foroptimum azimuthal and elevation resolution.

The transducer assembly 72 may also include a housing having the PMUTtransducer and associated circuitry disposed therein, where the housinghas an opening that ultrasound signals generated by the PMUT transducertravel through. In yet another alternative embodiment, the transducerassembly 72 includes an ultrasound transducer array (for example, arrayshaving 16, 32, 64, or 128 elements are utilized in some embodiments).

The rotation of the imaging core 62 within the sheath 60 is controlledby the interface module 54, which provides user interface controls thatcan be manipulated by a user. The interface module 54 can receive,analyze, and/or display information received through the imaging core62. It will be appreciated that any suitable functionality, controls,information processing and analysis, and display can be incorporatedinto the interface module 54. In an example, the interface module 54receives data corresponding to ultrasound signals (echoes) detected bythe imaging core 62 and forwards the received echo data to the controlsystem 56. In an example, the interface module 54 performs preliminaryprocessing of the echo data prior to transmitting the echo data to thecontrol system 56. The interface module 54 may perform amplification,filtering, and/or aggregating of the echo data. The interface module 54can also supply high- and low-voltage DC power to support operation ofthe catheter 52 including the circuitry within the transducer assembly72.

In some embodiments, wires associated with the IVUS imaging system 50extend from the control system 56 to the interface module 54 such thatsignals from the control system 56 can be communicated to the interfacemodule 54 and/or vice versa. In some embodiments, the control system 56communicates wirelessly with the interface module 54. Similarly, it isunderstood that, in some embodiments, wires associated with the IVUSimaging system 50 extend from the control system 56 to the monitor 58such that signals from the control system 56 can be communicated to themonitor 58 and/or vice versa. In some embodiments, the control system 56communicates wirelessly with the monitor 58.

FIG. 2 is a schematic drawing depicting a medical sensing system 100including a multi-modality processing system 101 according to oneembodiment of the present disclosure. In general, the medical sensingsystem 100 provides for coherent integration and consolidation ofmultiple forms of acquisition and processing elements designed to besensitive to a variety of methods used to acquire and interpret humanbiological physiology and morphological information. More specifically,in system 100, the multi-modality processing system 101 is an integrateddevice for the acquisition, control, interpretation, and display ofmulti-modality medical sensing data. In one embodiment, the processingsystem 101 is a computer workstation with the hardware and software toacquire, process, and display multi-modality medical sensing data, butin other embodiments, the processing system 101 may be any other type ofcomputing system operable to process medical sensing data. In theembodiments in which processing system 101 is a computer workstation,the system includes at least a processor such as a microcontroller or adedicated central processing unit (CPU), a non-transitorycomputer-readable storage medium such as a hard drive, random accessmemory (RAM), and/or compact disk read only memory (CD-ROM), a videocontroller such as a graphics processing unit (GPU), and a networkcommunication device such as an Ethernet controller.

In the illustrated embodiment, the medical sensing system 100 isdeployed in a catheter lab 102 having a control room 104, with theprocessing system 101 being located in the control room. In otherembodiments, the processing system 101 may be located elsewhere, such asin the catheter lab 102 itself as will be described in association withFIGS. 3 and 4. The catheter lab 102 includes a sterile field but itsassociated control room 104 may or may not be sterile depending on therequirements of a procedure and/or health care facility. The catheterlab and control room may be used to perform on a patient any number ofmedical sensing procedures such as angiography, intravascular ultrasound(IVUS), virtual histology (VH), forward looking IVUS (FL-IVUS),intravascular photoacoustic (IVPA) imaging, a fractional flow reserve(FFR) determination, a coronary flow reserve (CFR) determination,optical coherence tomography (OCT), computed tomography, intracardiacechocardiography (ICE), forward-looking ICE (FLICE), intravascularpalpography, transesophageal ultrasound, or any other medical sensingmodalities known in the art. For example, in catheter lab 102 a patient106 may be undergoing a multi-modality procedure either as a singleprocedure or in combination with one or more sensing procedures, inwhich IVUS data will be collected with an IVUS catheter 108 (which maybe implemented as an embodiment of the IVUS catheter 52 of FIG. 1), andOCT data will be collected with an OCT catheter 110. In someembodiments, the IVUS catheter 108 may also include one or more sensorssuch as a phased-array transducer and may be capable of multi-modalitysensing such as IVUS and IVPA sensing. The OCT catheter 110 may includeone or more optical sensors.

In the embodiment illustrated in FIG. 2, the medical sensing system 100includes a number of interconnected medical sensing-related tools in thecatheter lab 102 and control room 104 to facilitate this multi-modalityworkflow procedure, including an IVUS patient isolation module (PIM)112, an OCT PIM 114, an electrocardiogram (ECG) device 116, an angiogramsystem 117, a bedside control surface 118, a control room controlsurface 120, and a boom display 122. A bedside utility box (BUB) 124 inthe catheter lab 102 acts as a hub for the PIMs 112 and 114, ECG device116, and bedside control surface 118 and communicatively couples them tothe processing system 101. In the illustrated embodiment, theangiography system 117, control room control surface 120, and boomdisplay 122 are communicatively coupled directly to the processingsystem 101. However, in alternative embodiments, these tools may becoupled to the processing system 101 via the BUB 124. In one embodiment,the BUB 124 is a passive cable pass-through device that consolidateswires and feeds them into an under-floor cabling trench, but,alternatively, in other embodiments, the BUB 124 may contain logic andcommunication circuitry to actively coordinate communication between themedical sensing tools and the processing system 101. U.S. ProvisionalPatent Application No. 61/473,625, entitled “MEDICAL SENSINGCOMMUNICATION SYSTEM AND METHOD” and filed on Apr. 8, 2011, discloses abedside utility box that intelligently couples medical sensing-relatedtools and is hereby incorporated by reference in its entirety. Further,the multi-modality processing system 101 is communicatively coupled to adata network 125. In the illustrated embodiment, the data network 125 isa TCP/IP-based local area network (LAN), however in other embodiments,it may utilize a different protocol such as Synchronous OpticalNetworking (SONET), or may be a wide area network (WAN). The processingsystem 101 may connect to various resources via the network 125. Forexample, the processing system 101 may communicate with a DigitalImaging and Communications in Medicine (DICOM) system 126, a PictureArchiving and Communication System (PACS) 127, and a HospitalInformation System 128 through the network 125.

Additionally, in the illustrated embodiment, medical sensing tools insystem 100 are communicatively coupled to the processing system 101 viaa wired connection such as a standard copper link or a fiber optic link,but, in alternative embodiments, the tools may be connected to theprocessing system 101 via wireless connections using IEEE 802.11 Wi-Fistandards, Ultra Wide-Band (UWB) standards, wireless FireWire, wirelessUSB, or another high-speed wireless networking standard.

In the medical sensing system 100, the IVUS PIM 112 and OCT PIM 114 areoperable to respectively receive medical sensing data collected from thepatient 106 by the IVUS catheter 108 and OCT catheter 110 and areoperable to transmit the received data to the processing system 101 inthe control room 104. In one embodiment, the IVUS PIM 112 and OCT PIM114 transmit the medical sensing data over a Peripheral ComponentInterconnect Express (PCIe) data bus connection, but, in otherembodiments, they may transmit data over a USB connection, a Thunderboltconnection, a FireWire connection, or some other high-speed data busconnection. In one embodiment, the PIMs 112 and 114 include analog todigital (A/D) converters and transmit digital data to the processingsystem 101, however, in other embodiments, the PIMs transmit analog datato the processing system. Additionally, the ECG device 116 is operableto transmit electrocardiogram signals or other hemodynamic data frompatient 106 to the processing system 101. In some embodiments, theprocessing system 101 may be operable to synchronize data collectionwith the catheters 108 and 110 using ECG signals from the ECG 116.Further, the angiogram system 117 is operable to collect x-ray, computedtomography (CT), or magnetic resonance images (MRI) of the patient 106and transmit them to the processing system 101. In one embodiment, theangiogram system 117 may be communicatively coupled to the processingsystem 101 through the network 125, but, in other embodiments, theangiogram system may be more directly coupled to the processing system101, for example through an adapter device. Such an adaptor device maytransform data from a proprietary third-party format into a formatusable by the processing system 101. In some embodiments, the processingsystem 101 may co-register image data from angiogram system 117 (e.g.x-ray data, MRI data, CT data, etc.) with sensing data from the IVUS andOCT catheters 108 and 110. As one aspect of this, the co-registrationmay be performed to generate three-dimensional images with the sensingdata.

The bedside control surface 118 is also communicatively coupled to theprocessing system 101 via the BUB 124 and provides user control of theparticular medical sensing modality (or modalities) being used todiagnose the patient 106. In the current embodiment, the bedside controlsurface 118 is a touch screen that provides user controls and diagnosticimages on a single surface. In alternative embodiments, however, thebedside control surface 118 may include both a non-interactive displayand separate controls such as physical buttons and/or a joystick. In theintegrated medical sensing system 100, the bedside control surface 118is operable to present workflow control options and patient image datain graphical user interfaces (GUIs). The bedside control surface 118 iscapable of displaying GUIs for multiple modalities, and thus a cliniciandoes not have to physically move between user interface devices whenswitching sensing modalities.

The control room control surface 120 in the control room 104 is alsocommunicatively coupled to the processing system 101 and, as shown inFIG. 2, is adjacent to catheter lab 102. In the current embodiment, thecontrol room control surface 120 is similar to the bedside controlsurface 118 in that it includes a touch screen and is operable todisplay multitude of GUI-based workflows corresponding to differentmedical sensing modalities. In some embodiments, the control roomcontrol surface 120 may be used to simultaneously carry out a differentaspect of a procedure's workflow than the bedside control surface 118.In alternative embodiments, the control room control surface 120 mayinclude a non-interactive display and standalone controls such as amouse and keyboard.

The system 100 further includes the boom display 122 communicativelycoupled to the processing system 101. The boom display 122 may includean array of monitors, each capable of displaying different informationassociated with a medical sensing procedure. For example, during an IVUSprocedure, one monitor in the boom display 122 may display a tomographicview and one monitor may display a sagittal view.

FIG. 3 is a schematic drawing depicting a medical sensing system 250including a multi-modality processing system 252 according to anotherembodiment of the present disclosure. FIG. 4 is a diagrammaticperspective view of the multi-modality processing system 252 of FIG. 3.Like the processing system 101 in system 100, the multi-modalityprocessing system 252 in medical sensing system 250 is an integrateddevice for the acquisition, control, interpretation, and display ofmulti-modality medical sensing data. In this regard, the processingsystem 250 contains a similar software framework as processing system101.

However, the processing system 252 is mobile and may be moved betweencatheter labs. In the illustrated embodiment, the processing system 250is currently located in catheter lab 102 to perform an IVUS and OCTworkflow on patient 106. In the medical sensing system 250, the IVUS PIM112, OCT PIM 114, ECG system 116, angiography system 117, boom display122, and data network 125 are communicatively coupled to the processingsystem 250. Although the medical sensing tools in system 250 are shownas communicatively coupled to each other and the processing system 252via a wired connection (e.g. standard copper link, a fiber optic link),the tools may be connected to the processing system 252 via wirelessconnections (e.g. IEEE 802.11 Wi-Fi, UWB, wireless FireWire, wirelessUSB) in other embodiments.

Further, as shown in FIG. 4, the processing system 252 sits on a wheeledcart 111 to facilitate the mobility of the system. In some embodiments,the cart 111 may include wire management systems to control the variouswires attached to the system 252. Additionally, a controller 256 sits onthe cart 111 and is communicatively coupled to the processing system252. The controller 256 is operable to present a GUI to a clinicianconducting a workflow. In the illustrated embodiment, the controller 256is a touch screen that provides user controls and diagnostic images on asingle surface. In alternative embodiments, however, the controller 256may include both a non-interactive display and separate controls such asphysical buttons and/or a joystick.

Although illustrative embodiments have been shown and described, a widerange of modification, change, and substitution is contemplated in theforegoing disclosure and in some instances, some features of the presentdisclosure may be employed without a corresponding use of the otherfeatures. For example, in some embodiments, the multi-modalityprocessing systems 101 and 252 may be used to process non-cardiovasculardiagnostic data such as data from cranial or peripheral arteries, aswell as data from non-vascular body portions. Further, the systems 101and 252 may be used to collect and process MRI or CT data, or may beutilized in computer assisted surgery (CAS) applications. Further, themodules described above in association with the multi-modalityprocessing systems may be implemented in hardware, software, or acombination of both. And the processing systems may be designed to workon any specific architecture. For example, the systems may be executedon a single computer, local area networks, client-server networks, widearea networks, internets, hand-held and other portable and wirelessdevices and networks.

FIGS. 1-4 discussed above offer an example medical context in which avirtual pullback of an intravascular imaging device may be performedaccording to the various aspects of the present disclosure. In moredetail, FIGS. 5-8 are screenshots of a user interface 300 for performingsuch virtual pullback of an intravascular imaging device. In certainembodiments, the user interface 300 may be implemented on the bedsidecontrol surface 118 (FIG. 2), the control room control surface 120 (FIG.2), or the controller 256 (FIGS. 3-4). A human user/operator, forexample a physician, may interactively engage with the user interface300 to perform the virtual pullback of an intravascular imaging device,as discussed below in more detail.

The user interface 300A shown in FIG. 5 contains a plurality of views310, 311, and 312. Each view 310-312 is shown in a respective virtualwindow of the user interface 300A. The views 310-312 collectively offera virtual illustration of what is happening in a part of a humananatomy, with different types of imaging, in response to an operator'sactions. The imaging content of the view 310 is different than theimaging content of the view 311. As such, it may be said that the views310-311 are configured to display different imaging modalities. Thesimultaneous display of multiple imaging modalities in different viewsis referred to as co-registration, a more detailed discussion of whichcan be found in U.S. Pat. No. 7,930,014, titled “VASCULAR IMAGECO-REGISTRATION” to Huennekens et al, and U.S. Pat. No. 8,298,147,titled “THREE-DIMENSIONAL CO-REGISTRATION FOR INTRAVASCULAR DIAGNOSISAND THERAPY” to Huennekens et al, the entire disclosures of which areherein incorporated by reference in their entirety.

The view 310 includes a virtual representation of an anatomical region320 of a human body. In the embodiment shown in FIG. 5, the anatomicalregion 320 is a blood vessel. However, the anatomical region 320 mayinclude other parts of the human anatomy, such as different parts of thevasculature or any portion of the circulatory system. In someembodiments, the virtual representation of the anatomical region 320 isgenerated from a computer model, such as a two-dimensional orthree-dimensional model of a typical human body. Such computer modelsmay be manipulated by the user/operator in various ways, includingmoving, scaling, resizing, rotating, etc. In other embodiments, thevirtual representation of the anatomical region 320 may be generated byactual imaging data, for example an angiogram, x-ray, MRI, CT, or othersuitable imaging data taken from an actual patient (e.g., the patient106 in FIGS. 2-3).

The view 310 also includes a virtual representation of an intravascularimaging device 330. In the embodiment shown in FIG. 5, the intravascularimaging device 330 is an IVUS catheter, for example the catheter 52 ofFIG. 1. The virtual representation of the intravascular imaging device330 is a substantially accurate representation of the actualintravascular imaging device 330. For example, the virtualrepresentation accurately depicts the shape and geometry of theintravascular imaging device 330. The virtual representation of theintravascular imaging device 330 also includes a plurality of markers340, illustrated as dots on the intravascular imaging device 330. Thesemarkers 340 denote units of length. For example, the distance betweenadjacent markers 340 may represent one centimeter (or another suitableunit of length). In this manner, the user/operator may gauge thedimensions of the anatomical region 320 relative to the intravascularimaging device 330. It is understood, however, that the markers 340 arenot necessarily required, and they may or may not be implemented inalternative embodiments.

The virtual representation of the intravascular imaging device 330 alsoincludes a virtual representation of an ultrasound transducer 350.Again, the virtual representation of the ultrasound transducer 350 mayaccurately depict the shape and geometry of the actual ultrasoundtransducer 350. The ultrasound transducer 350 is configured to capturecross-sectional ultrasound images of the anatomical region 320. In moredetail, as the intravascular imaging device 330 is being moved withrespect to the anatomical region 320 (for example being pulled out), theultrasound transducer 350 continuously takes cross-sectional images ofthe anatomical region 320. Each cross-sectional ultrasound imagecorresponds to a particular location of the ultrasound transducer 350inside the anatomical region 320. These ultrasound images are stored ina suitable computer data system, for example in the multi-modalityprocessing system 101 of FIG. 2.

According to the various aspects of the present disclosure, thecross-sectional images are displayed in the view 311. Each displayedultrasound image in the view 311 corresponds to the particular locationof the virtual representation of the ultrasound transducer 350. The view311 updates the displayed ultrasound images as the intravascular imagingdevice 330 is being virtually pulled back.

In the embodiment shown in FIG. 5, the view 312 is being used tofacilitate the virtual pullback of the intravascular imaging device 330.In more detail, the view 312 shows another virtual representation of theintravascular imaging device 330, which also contains markers 340 (notshown in FIG. 5 but shown in FIG. 6) that denote units of length. Thoughthe displayed sizes between the virtual representations of theintravascular imaging device 330 may vary between the two views 310 and312, it is understood that they represent the same underlying actualdevice. Thus, as the virtual representation of the intravascular imagingdevice 330 is being pulled out of the patient body in the view 312, thevirtual representation of the intravascular imaging device 330 is alsopulled back in the same manner in the view 310. The amount of thepullback is the same in both views 310 and 312. For example, if thevirtual representation of the intravascular imaging device 330 is pulledby three markers in the view 312, then the virtual representation of theintravascular imaging device 330 is also pulled by three markers in theview 310.

In the embodiment shown in FIG. 5, the view 312 utilizes a virtual hand360 to perform the virtual pullback of the intravascular imaging device330. Of course, other suitable virtual mechanisms may also be utilizedin some embodiments to accomplish the virtual pullback of theintravascular imaging device 330.

Referring now to FIG. 6, to provide a visual example of the discussionsabove, the user interface 300B illustrates another snapshot of the threedifferent views 310-312. The user/operator may utilize the virtual hand360 to pull back the virtual representation of the intravascular imagingdevice 330 in the view 312. For example, in embodiments where the userinterface 300B is implemented on a touch-sensitive screen, theuser/operator may user his/her finger(s) to touch and drag the virtualhand 360 to the right direction so as to pull on the virtualrepresentation of the intravascular imaging device 330. In otherembodiments where the user interface 300B is implemented on anon-touch-sensitive display, then a mouse or some other type of userinput/output mechanism may be used to engage the virtual hand 360 topull on the virtual representation of the intravascular imaging device330.

As the virtual representation of the intravascular imaging device 330 ispulled in the view 312, it is also pulled in the view 310 in asubstantially simultaneous manner (e.g., a delay of a number ofmilliseconds may be acceptable, particularly if such delay is notperceived by the user/operator). The user/operator may see in a clearvisual manner where the intravascular imaging device 330 is relative tothe anatomical region 320 throughout the virtual pullback process.Meanwhile, the ultrasound image in the view 311 is also continuouslyupdated as the intravascular imaging device 330 is being virtuallypulled. Again, each displayed ultrasound image in the view 311corresponds to a location of the ultrasound transducer 350 inside theanatomical region 320.

Based on the above discussions, it can be seen that the interactive userinterface 300 can be used as a powerful diagnostic tool. In someembodiments, an accurate virtual model is built for an intravascularorgan (e.g., a particular blood vessel) of interest for a particularpatient. In other words, this virtual model is customized for thatspecific patient. Such virtual model may be generated based on anangiogram, for example. In other embodiments, a more generic virtualmodel may be established, which may not be customized for any singlepatient. In any case, such virtual model is represented by theanatomical region 320 shown in the view 310 of the user interface 300.An actual intravascular imaging device (such as a catheter) is insertedinto the intravascular organ of the target patient, and an actualpullback process of the intravascular imaging device is performed. Asthe actual pullback process takes place, ultrasound images at differentlocations of the intravascular organ are taken and recorded.

At a later time, the user/operator (which may be a physician ordiagnostician) may perform the virtual pullback process of theintravascular imaging device 330 as discussed above, for exampleengaging the virtual hand 360 to pull on the intravascular imagingdevice 330. The virtual pullback process in a way simulates the actualpullback of the intravascular imaging device 330. During the virtualpullback, the user/operator may spot problem areas in one or moreultrasound images displayed in the view 311. When this occurs, theuser/operator may refer to the view 310 and see the exactly location(s)in the anatomical region 320 that corresponds to the problematicultrasound image(s). The availability of such locational information mayallow the user/operator to perform better diagnoses or perform bettermedical procedures.

As discussed above, the virtual hand 360 is not the only availablemechanism for performing the virtual pullback process. Referring now toFIGS. 7-8, another embodiment of the user interface to perform thevirtual pullback is illustrated. In FIG. 7, the user interface 300Cstill includes the views 310 and 311, but does not need the view 312that contains the virtual hand. Instead, the user interface 300Cutilizes a virtual control mechanism 380 in the view 310 to facilitatethe virtual pullback process.

In the embodiment shown in FIG. 7, the virtual control mechanism 380 isa slider, where an indicator 390 inside the slider 380 can be moved upor down by a touch-sensitive engagement or by a mouse or anothersuitable input/output device. The total length of the slider 380 maycorrespond to the total length of the intravascular imaging device 330.As the indicator 390 is moved up or down in the slider 380, the virtualrepresentation of the intravascular imaging device 330 is also moved upor down inside the anatomical region 320. Therefore, the slider 380 andthe indicator 390 may be used to accomplish the same task as the virtualhand 360 in FIGS. 5-6, which is to induce the virtual pullback of theintravascular imaging device 330.

For example, in FIG. 8, the indicator 390 is moved downwards in theslider 380. Correspondingly, the virtual representation of theintravascular imaging device 330 is also pulled downwards. Meanwhile,the ultrasound image displayed in the view 311 is updated to reflect thenew position of the intravascular imaging device 330 inside theanatomical region 320. Again, the continuously updated display of theultrasound image in the view 311 to reflect the positional movement ofthe intravascular imaging device 330. It is understood that othersuitable virtual control mechanisms may be implemented in alternativeembodiments to facilitate the virtual pullback process discussed above,but they are not discussed herein for reasons of simplicity.

The embodiments discussed above involve using the virtual pullbackprocess for the diagnosis and/or performing medical procedures for anactual patient. However, this is not the only use of the virtualpullback process. In some embodiments, the virtual pullback process mayalso be used for mainly educational purposes. For example, in someembodiments, the user interface 300 (or something similar that allowsfor a demonstration of the virtual pullback process) may be implementedon an electronic tablet computer, such as an iPad, an iPhone, or othersmart phones or tablets. The virtual representations of the anatomicalregion 320 or the intravascular imaging device 330 may not necessarilybe tied to an actual patient or an actual product, but may simply bemodels for demonstration purposes. Nevertheless, the performance of thevirtual pullback process as discussed above will still allow theuser/operator to simulate the experience of a virtual pullback in thereal world medical environment. That is, the ultrasound images are stillcontinuously updated to reflect the movement of the intravascularimaging device.

In yet other embodiments, the virtual pullback process discussed abovemay be implemented in a more game-like environment. For example, amedical simulation game may be created that utilizes the user interface300 discussed above. An example objective of the game may be to find adiseased area of a patient (e.g., by observing the ultrasound images andthe corresponding virtual representation of the anatomical region).Another example objective of the game may be to perform one or moremeasurements in the anatomical region, which may involve using thevirtual markers discussed above. To further enhance the gaming aspect ofthe virtual pullback process and to make the games more challenging,these game objectives may also need to be completed within apredetermined amount of time.

It is also understood that the virtual pullback process discussed abovemay be implemented on any suitable electronic device, for example anelectronic tablet such as the iPad or an Android-powered tablet,smartphones such as iPhones, Android-powered smartphones, windowsphones, blackberries, laptop or desktop computers, or suitable medicaldevices and/or systems from a medical device manufacturer (e.g., the s5imaging system from Volcano, Corp.). To the extent applicable, the touchscreens of these electronic devices may be used as the touch-sensitivecontrol mechanism on which an operator can perform the virtual pullbackprocess. Alternatively, other input/output means such as mouses orkeyboards may also be used to perform the virtual pullback process.

The virtual pullback process may also be used in medical contexts wherea physician may be remotely located. For example, some robotics-assistedmedical systems allow a physician to be situated remotely from a patientbut still able to place or pullback a catheter (or other intravenousmedical devices) in a target patient. In these scenarios, theremotely-located physician may be able to perform a virtual medicalprocedure similar to the virtual pullback process discussed above on aninteractive device (e.g., a touch screen), and the robotics of themedical system may perform the actual medical procedure based on theinput received on the interactive device according to the physician'sactions. In other words, the robotics involved may simulate thephysician's actions performed in a virtual context.

It is also understood that the user interface 300 discussed above may beconfigured to optionally display various interactions with a giventherapy. For example, the user interface may be configured to show thestent or treatment device position before or after its deployment. Othersuitable interactions are envisioned but are not discussed herein forreasons of simplicity.

FIG. 9 is a flowchart of a method 600 for simulating an intravascularprocedure in a virtual environment according to various aspects of thepresent disclosure. The method 600 includes a step 610, in whichinformation is displayed from a first view and a second viewsimultaneously. The first view contains virtual representations of ananatomical region of a human body and an intravascular imaging devicedisposed in the anatomical region. The second view contains across-sectional image of a segment of the anatomical regioncorresponding to a location of the intravascular imaging device.

The method 600 includes a step 620, in which the virtual representationof the intravascular imaging device is moved with respect to the virtualrepresentation of the anatomical region. The step 620 is performed inresponse to a user input.

The method 600 includes a step 630, in which the cross-sectional imageis updated as the virtual representation of the intravascular imagingdevice is being moved. The updated cross-sectional image corresponds toa new location of the intravascular imaging device.

It is understood that additional steps may be performed to complete thesimulation of an intravascular procedure in a virtual environment.However, these additional steps are not discussed herein for reasons ofsimplicity.

Persons skilled in the art will recognize that the apparatus, systems,and methods described above can be modified in various ways.Accordingly, persons of ordinary skill in the art will appreciate thatthe embodiments encompassed by the present disclosure are not limited tothe particular exemplary embodiments described above. In that regard,although illustrative embodiments have been shown and described, a widerange of modification, change, and substitution is contemplated in theforegoing disclosure. It is understood that such variations may be madeto the foregoing without departing from the scope of the presentdisclosure. Accordingly, it is appropriate that the appended claims beconstrued broadly and in a manner consistent with the presentdisclosure.

What is claimed is:
 1. A method of simulating an intravascular procedurein a virtual environment, the method comprising: displaying informationfrom a first view and a second view simultaneously, wherein the firstview contains virtual representations of an anatomical region of a humanbody and an intravascular imaging device disposed in the anatomicalregion, and wherein the second view contains a cross-sectional image ofa segment of the anatomical region corresponding to a location of theintravascular imaging device, and wherein the virtual representation ofthe intravascular imaging device includes an accurate visual portrayalof a shape and a geometry of the intravascular imaging device, includingan elongate catheter body, a transducer, and a plurality of markerslocated along at least a portion of a longitudinal axis of the catheterbody, and wherein the virtual representation of the intravascularimaging device has a first transducer position with respect to theanatomical region and a first number of the markers; moving, in responseto a user input, the virtual representation of the intravascular imagingdevice with respect to the virtual representation of the anatomicalregion such that, as the virtual representation of the intravascularimaging device is being moved, it is being displayed with a plurality ofsecond transducer positions with respect to the anatomical region thatare different from the first transducer position and a plurality ofsecond numbers of the markers that are different from the first numberof the markers, wherein the moving comprises covering up an increasingnumber of the markers at least in part by the transducer as theintravascular imaging device is being moved out of the anatomicalregion; and updating the cross-sectional image as the virtualrepresentation of the intravascular imaging device is being moved,wherein the updated cross-sectional image corresponds to a new locationof the intravascular imaging device.
 2. The method of claim 1, whereinthe anatomical region comprises a part of a circulatory system.
 3. Themethod of claim 1, wherein the virtual representation of the anatomicalregion comprises a three-dimensional computer model of the anatomicalregion.
 4. The method of claim 1, wherein the virtual representation ofthe anatomical region comprises an angiogram of the anatomical region ofan actual patient.
 5. The method of claim 1, wherein the cross-sectionalimage comprises an ultrasound image.
 6. The method of claim 1, whereinthe markers denote units of length and are evenly spaced apart from oneanother.
 7. The method of claim 1, wherein the displaying furthercomprises displaying a third view simultaneously with the first view andthe second view.
 8. The method of claim 7, wherein: the virtualrepresentation of the intravascular imaging device is a first virtualrepresentation of the intravascular imaging device; and the third viewcontains a second virtual representation of the intravascular imagingdevice and a virtual control mechanism for pulling the second virtualrepresentation intravascular imaging device in response to the userinput.
 9. The method of claim 8, wherein a movement of the first virtualrepresentation of the intravascular imaging device corresponds to apulling of the second virtual representation of the intravascularimaging device.
 10. The method of claim 8, wherein the virtual controlmechanism comprises a virtual representation of a human hand, whereinthe displaying comprises displaying, in the third view, a medicalenvironment in which the second virtual representation of theintravascular imaging device is pulled out of the human body by thehuman hand.
 11. The method of claim 1, wherein the first view furthercomprises a virtual control mechanism for moving the virtualrepresentation of the intravascular imaging device in response to theuser input.
 12. The method of claim 1, wherein the displaying isperformed using a touch-sensitive screen, and wherein the user input isreceived through the touch-sensitive screen.
 13. The method of claim 1,wherein the first view and the second view are configured to displaydifferent imaging modalities.
 14. An electronic apparatus configured toperform a virtual pullback of an intravascular imaging device, theelectronic apparatus comprising: a screen configured to display anoutput to the user; a memory storage component configured to storeprogramming code; and a computer processor configured to execute theprogramming code to perform the following tasks: displaying, on thescreen, information from a first view and a second view simultaneously,wherein the first view contains virtual representations of an anatomicalregion of a human body and an intravascular imaging device disposed inthe anatomical region, and wherein the second view contains across-sectional image of a segment of the anatomical regioncorresponding to a location of the intravascular imaging device, andwherein the first virtual representation of the intravascular imagingdevice includes an accurate visual portrayal of a shape and a geometryof the intravascular imaging device, including an elongate catheterbody, a transducer, and a plurality of markers located along at least aportion of a longitudinal axis of the catheter body, and wherein thevirtual representation of the intravascular imaging device has a firsttransducer position with respect to the anatomical region and a firstnumber of the markers; moving, in response to a user input, the virtualrepresentation of the intravascular imaging device with respect to thevirtual representation of the anatomical region such that, as thevirtual representation of the intravascular imaging device is beingmoved, it is being displayed with a plurality of second transducerpositions with respect to the anatomical region that are different fromthe first transducer position and a plurality of second numbers of themarkers that are different from the first number of the markers, whereinthe moving comprises covering UP an increasing number of the markers atleast in part by the transducer as the intravascular imaging device isbeing moved out of the anatomical region; and updating thecross-sectional image as the first virtual representation of theintravascular imaging device is being moved, wherein the updatedcross-sectional image corresponds to a new location of the intravascularimaging device.
 15. The electronic apparatus of claim 14, wherein theanatomical region comprises a part of a circulatory system.
 16. Theelectronic apparatus of claim 14, wherein the virtual representation ofthe anatomical region comprises a three-dimensional computer model ofthe anatomical region.
 17. The electronic apparatus of claim 14, whereinthe virtual representation of the anatomical region comprises anangiogram of the anatomical region of an actual patient.
 18. Theelectronic apparatus of claim 14, wherein the cross-sectional imagecomprises an ultrasound image.
 19. The electronic apparatus of claim 14,wherein the intravascular imaging device comprises a catheter with atransducer implemented thereon markers denote units of length and areevenly spaced apart from one another.
 20. The electronic apparatus ofclaim 14, wherein the displaying further comprises displaying a thirdview simultaneously with the first view and the second view.
 21. Theelectronic apparatus of claim 20, wherein: the virtual representation ofthe intravascular imaging device is a first virtual representation ofthe intravascular imaging device; and the third view contains a secondvirtual representation of the intravascular imaging device and a virtualcontrol mechanism for pulling the second virtual representationintravascular imaging device in response to the user input.
 22. Theelectronic apparatus of claim 21, wherein a movement of the firstvirtual representation of the intravascular imaging device correspondsto a pulling of the second virtual representation of the intravascularimaging device.
 23. The electronic apparatus of claim 21, wherein thevirtual control mechanism comprises a virtual representation of a humanhand, wherein the displaying comprises displaying, in the third view, amedical environment in which the second virtual representation of theintravascular imaging device is pulled out of the human body by thehuman hand.
 24. The electronic apparatus of claim 14, wherein the firstview further comprises a virtual control mechanism for moving thevirtual representation of the intravascular imaging device in responseto the user input.
 25. The electronic apparatus of claim 14, wherein thescreen is a touch-sensitive screen, and wherein the user input isreceived through the touch-sensitive screen.
 26. The electronicapparatus of claim 14, wherein the first view and the second view areconfigured to display different imaging modalities.
 27. An apparatuscomprising a non-transitory, tangible machine-readable storage mediumstoring a computer program, wherein the computer program containsmachine-readable instructions that when executed electronically bycomputer processors, perform: displaying, on a touch-sensitive screen,information from a first view and a second view simultaneously, whereinthe first view contains virtual representations of an anatomical regionof a human body and an intravascular imaging device disposed in theanatomical region, and wherein the second view contains across-sectional image of a segment of the anatomical regioncorresponding to a location of the intravascular imaging device, andwherein the virtual representation of the intravascular imaging deviceincludes an accurate visual portrayal of a shape and a geometry of theintravascular imaging device, including an elongate catheter body, atransducer, and a plurality of markers located along at least a portionof a longitudinal axis of the catheter body, and wherein the virtualrepresentation of the intravascular imaging device has a firsttransducer position with respect to the anatomical region and a firstnumber of the markers; moving, in response to a user input, the virtualrepresentation of the intravascular imaging device with respect to thevirtual representation of the anatomical region such that as the virtualrepresentation of the intravascular imaging device is being moved, it isbeing displayed with a plurality of second transducer positions withrespect to the anatomical region that are different from the firsttransducer position and a plurality of second numbers of the markersthat are different from the first number of the markers, wherein themoving comprises covering up an increasing number of the markers atleast in part by the transducer as the intravascular imaging device isbeing moved out of the anatomical region; and updating thecross-sectional image as the virtual representation of the intravascularimaging device is being moved, wherein the updated cross-sectional imagecorresponds to a new location of the intravascular imaging device. 28.The apparatus of claim 27, wherein: the anatomical region comprises apart of a circulatory system; the virtual representation of theanatomical region comprises a three-dimensional computer model of theanatomical region or an angiogram of the anatomical region of an actualpatient; the cross-sectional image comprises an ultrasound image; andthe markers denote units of length and are evenly spaced apart from oneanother.
 29. The apparatus of claim 27, wherein: the instructions forthe displaying further comprise instructions for displaying a third viewsimultaneously with the first view and the second view; the virtualrepresentation of the intravascular imaging device is a first virtualrepresentation of the intravascular imaging device; and the third viewcontains a second virtual representation of the intravascular imagingdevice and a virtual control mechanism for pulling the second virtualrepresentation intravascular imaging device in response to the userinput.
 30. The apparatus of claim 29, wherein: a movement of the firstvirtual representation of the intravascular imaging device correspondsto a pulling of the second virtual representation of the intravascularimaging device; and the virtual control mechanism comprises a virtualrepresentation of a human hand, wherein the displaying comprisesdisplaying, in the third view, a medical environment in which the secondvirtual representation of the intravascular imaging device is pulled outof the human body by the human hand.
 31. The apparatus of claim 27,wherein the first view further comprises a virtual control mechanism formoving the virtual representation of the intravascular imaging device inresponse to the user input.